Field of the Invention
The present invention relates to an image processing technique and, more particularly, to control when wirelessly transmitting an image.
Description of the Related Art
In recent years, there has been known an MR (Mixed Reality) technique of superimposing and displaying a video in a virtual world expressed by a CG (Computer Graphics) image on a video in a real world. As an apparatus which uses the MR technique, there is provided a video see-through type HMD (Head Mounted Display). The video see-through type HMD causes a video camera or the like to capture a video, which nearly coincides with that viewed from a pupil position of an HMD user, and superimposes and displays a CG image on the captured image.
The video see-through type HMD transmits, to an image processing apparatus such as a PC (Personal Computer), an image captured by an image capturing unit mounted in the HMD, and the PC performs the rendering processing of a CG image based on the obtained captured image. The PC transmits the rendered CG image to the HMD, and the HMD displays an image obtained by compositing the captured image and the CG image, thereby allowing the user wearing the HMD to experience an MR space. The rendering processing of the CG image is performed based on the position and orientation information of the HMD. The position and orientation information of the HMD can be obtained based on feature points extracted using markers or natural features in the captured image.
In a wireless MR system in which an HMD and a PC are wirelessly, communicably connected, the HMD wirelessly transmits, to the PC, a captured image and information such as the position and orientation information of the HMD. The PC renders a CG image based on the captured image and the position and orientation information of the HMD, and wirelessly transmits the rendered CG image to the HMD.
Consequently, in the wireless MR system, a delay caused by wireless transmission between the HMD and the PC generates a delay until the MR image is displayed on the HMD after the HMD performs image capturing. The delay time generated by wireless transmission changes depending on the effective transmission rate of wireless communication and the data amounts of the captured image and CG image to be wirelessly transmitted. Furthermore, since the effective transmission rate of wireless communication changes depending on a radio wave propagation environment, the delay time generated by wireless communication always varies. If the delay time is long, there is a large difference between the timing of an actual operation and that of the MR image displayed on the HMD, and thus the user feels an unnatural impression, thereby degrading the feeling of the user as if he/she existed in the MR space.
To cope with this, Japanese Patent Laid-Open No. 2010-258850 (patent literature 1) discloses a technique of calculating a wireless transmission band based on a delay time and packet loss rate between a video transmission terminal and a video reception terminal, and determining a compression ratio in data transmission based on the calculated wireless transmission band.
The delay until the MR image is displayed on the HMD after the HMD performs image capturing can be generated by the rendering processing of the CG image by the PC. The time taken to execute the rendering processing of the CG image changes depending on, for example, the resolution of the CG image to be rendered, the size of the CG image, and the geometrical shape of the CG image.
However, the above-described conventional technique cannot cope with the delay time if the time of the rendering processing of the CG image is long. That is, in the method described in patent literature 1, even if the time of the rendering processing of the CG image is long, if the delay time in wireless transmission is short, it is impossible to reduce the delay time.